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ABSTRACT 

The focus of this paper is on the methodology used in 
analyzing studies on the effects of school desegregation on black 
achievement. The paper also addresses a number of substantive issues, 
including the overall effectiveness of school desegregation, the 
impact of type of achievement (math or reading), and time of 
desegregation (early or later grades). In the first section, a 
discussion of the advantages and disadvantages of meta-analysis is 
presented. The complications are explored of using the method to 
analyze the research literature on the effectiveness of school 
desegregation when that literature is almost totally composed of 
quasi-experimental or weaker research designs. In the second section,, 
the adjustments that were made in the meta-analysis method for the 
purposes of the National Institute of Education (NIE) panel's study 
on the effects of school desegregation are described. The procedures 
and criteria used for including the 31 studies finally chosen for 
analysis by the author are described in the third section, as well 
the criteria used by NIE in narrowing the selection down to 19 
studies for consideration by the panel. In section 4, the results of 
analyses of both sets of studies are given. An appendix lists the 
studies used in both analyses. (CMG) 
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1.0 PROBLEM 

R.e. r.l.tlon. b,t-..n bl.ck. dnd "hit.. h.v, pUyed . H.nlflc.nt 
r o«. .n th. hl.tory of th. United St.t... Socl.l th.ory .nd 

d,t., in prtlcuUr. km fi 9 ur.d pro»In.nt!y in th. eontrov.r.i., th.t 
h.v. con.t.ntly .urround.d wjor ov.nt. In th!« hi.tory. for «»Pl>. 
th. two LndMrk U.S. Supr.n, e Court d.ci.ion. d..1ln, wit* 
d... 9 r...t«»n, PL.«y v. f.r 9 u«.n in II*. .nd Bro-n v. Bo.rd of 
Eduction in HSU <K.u„.r, 1975) «r. both V...d In p.rt on curr.nt 
...... .clone. .vid.nc nor. r.cntly'. tn. .o-c.ll.d dmn R.port or 

tn. Equ.lit, of Edue.tlon.l Opportunity Surv.y (Co>».n. C.pb.11. 
Hob.on. ncP.rtl.nd. nood. ..inf.ld . York. 19«) »» u..d by th. John.on 
.d-.ini.tr.tion to .cc.l.r.t. th. d...»r.«tion proc... (Gr.nt. 1*973). 
Th. C.1«.h R.port .HM'thM b.«k .tud.nt .chUv.rn.nt incr..,.d in 
„or. int.,r.t.d .nvi ro«n.nt. (i.... -Ith . .r..t.r proportion of -hit. 

,tud.nt,). Thi. .tudy .nd findin, not only l.d to . ~»b.r of 
r..n.,y... by ..ci.l .ci.nti.t.. but o,.o to .n lncr..,in 9 nu»d.r of 
. y .tM..tic .tudU, u.lh 9 b.for. .nd .ft.r »...-«» '' 
po.tt..t.) of .Ci.v.a.nt .nd control or co.np.ri.on .roup, of .. 9 r. 9 .t.d 
buck.. Th... .tudi., .i».d .t .li»in.tin 9 th. m .thodc1o 9 ic.l 

„.kn.„., of cr.„-..etl.n.l .urv.y. .uch .. th. d« Mp"t - 
mtin, .on,, of It. hypoth.... .nd th... of oth.r .ocl.l .ci.nti.t,. 

, y th. .ld-1970'. th.r. h.d .cc»ul.t.« . .uffici.nt bod, of 
.C.ntif ic: .tudi.. th.t . nu^b.r of c.r.ful r.vi.w. .pp..r.d. T« of 

th. «.t not*?. 'if tn... r.vi c.nduct.d b, Br.dl.y .nd Br.d-.y 

(1977 , .nd St. John 0975). Th. Br.dl.y. .x.n-in.d W .tudi., of tn. 
.,f.ct. of d... 9 r. 9 .f.on on bUck .cnj.vw.nt -nil. St. ,.nn r.vi.^c tt 
(includin, » 2 cro..-..ct..n.l .tudi..). Both found th. .v.d.nc. 
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lnconclu.lv.. Th. Br,d..y. concluded th.t th. .vid.ne. on th. 
,f„ctW.n„. =f deselection on bl.e. .ehi.vement «.. "Inconsistent 
ond ln .d.qu.t.» whil. S,. John slmil.rly .Ocnowl.d 9 .d, "nor. th.n . 
dsc.de of consider.... r....,'.h effort hes produced no definitive 
positive finding." St. John went on to quote Li 9 ht end Smith 097D 
th.t •"pro.r.s. will only co« -h.n w. .r. eble to poo,, in . ..static 
m.nner. the orl 9 I«.l d.t. from the -Studies.- Such methods for 

. yl ,th..1x.«9 t Us of sc.ent.f.c stud.es h.v. recently 9 .i«.d 

widespre.d popularity ..r S e,y due to GU..' ..mln.- -or. on "»«.- 
analys is 

«et,-.nelysi« offer. . number of .dv.nt. 9 .s over previous methods 
for . 99 re 9 stin 9 th. findings of different studies (light t Smith. .971: ' 
Cess. .977). lnT.b,..w. h.v. iist.d of th. positiv. .nd 

M9 , ti ct.ri.tic. of thi. t.chnl,u.. Th. m.jor positive 

qu.lities.rc.sin,... prsci... q u.nt«t.t.v. ....ur. of th. .v.r. 9 . 
„.,nitud. of pro 9 r.m i.p«t. 

rMMftt end provide, .n import.nt resu.t th.t i, ...y to pr.sp. «.»- 
ooo.y.U ..so .Lows on. to consider ..mp.e si» .nd design que. ity. 
This technique ...o h.s its »d 1 ,.dv.«t. 9 .." .speci.i.y -hen extended to 
»tudi.s with m.thod.logic.1 problem, such .s . : u..i-.xp.r.».nt. (i.e.. 
studies locking random oss ignment) . 

' St.nd.rd met.-.n.lytic methods hove .lr..dy been .pplied to thi, 
.,t.r.tur. (Cr.in < na .982= Kro.. .978). 

performed byKro, and Cr.in .nd h.h.rd both found .rna.i positiv. 

benefits for dese 9 re 9 ,tion on bl.c. bchi. t. Mb .nd. .08 st.nd.rd 

d.vi.tions. resp.ctive.y). Bother. fU-d in our opinion. Kroi's 
,t„dy il.ustr.tes' th. in.ppropri.t. .pp. ic.tion of G..S.' «.thod. for 
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Table 1 




Meta^aiyAl* ggAgog 
ThlTaverage effect si*e 
of a hypothesis tested 
in many studies. The 
term connotes "the analy- 
sis of analyses. 1 .•- . 
the statistical analysis 
of the findings of many 
individual analyses." 



o Precise determination 
of effects 

6 Systematic, statistical 
approach 

6 Design quality can be 
examined 

o Can examine effect 
of sample alto 

o Includes some descriptive 
informat ion 



o susceptible to publication bias 
o Requires a control group 
o Requires statistical Information 

o Assumes a •common 
metric* for measure 

o Assumes the -strategic 
combination argument" 



•Adapted from Krol (1978) 
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.a*... CU» (1977. P- >5« *- '•»— P«-" peri "* M " 

contfoU "if the tr..t.d .roup o-~.' pretr..t».nt 
.ft-, i. . ..cd «t.».t. of their hypothetic.. po.ttr..».nt in the 
of tre.to.nt." ». we «... deoon.tr.t. In the next »ct..n. th.. 

, us9 .,t.on o.y»e . M ed.ndi..-dv.,.d. Cr.in «- «*» 

„ , very r.cent oet.-.n..,.i. h,v. «*. . tr.d,ti.n., G,.....n 
.ppro.ch .nd inc.uded ... .«ud... .« . «.„.!.. »» « 

indie . t e be.c thi. .PP-ch >• in.ppropri.te. .tudies 

h , v . „ B ,ny ..thodo.o.ic., we. k n...« th.t they -Hou.d not o. .nc.uded. 

(tofeover . ,o m e . t udU. -.-c h - .. tho n .-""'or... .urv.y 

c.nnot rU.d the n.e.,..ry .t.ti.t.e., infection (.inc. th.y both 

a pre-de,e 9 r. 9 .tion or pretest o... " " • — «'° UP) ' " 

n c.ud.d by Cr.in .nd Mh.ro. Otn.r .tud... used wh.t. con.ro, 

9rou ps or n.t.on.. test norms to p.ner.t. .ff.ct ~ both .r. 

lwW „.t. comp.ri.on. „i . . be d..cu.,.d be.o- Such .tudie. .- 

eeou nt for h..f of tho.. .nc.ud.d In Cr..n .nd MM*'. —«-»».«.. 

.oport.nt.y. ho-ov.r both Kro. .nd Cr..n .nd n.h.rd p..d 
i „ suf f.c..nt.tt.nt.ontoth.thr..t.tov.,.d.t y th.t cou.d confound 

.nd bi.s th. result, of th.ir met.- .n.ly..«. 

Th. .cnoo, d .,.,r. 9 .t.on-..hi.v...nt Ut.r.ture poses some .P«..< 

prob.ems for th. *.t.-.n..y... — . 

in coition .nd thu. .u.c.ptib.e to other 
"irpr.t,tion. (i.e.. ,.-c„>.d "P..u.ib.e r.v., hypothe.e.", . «.t- 
. Mly . 1 .of-.uch.tudi.....u». th.t e.ther .opropri.t. .t.ti.t.C 

' • * ,uKf»»t« to validity 1 ' or that 
adjustments can be made for the var.ous threats 

■, Miration argument" (Staines. 197« <^ os ise£ 
tne "strategic combination «'* 

.« m Table 1). This latter term stands for the belief 
"disadvantages'- m Table u . 



that flawed studies can be combined because the "weaknesses cancel each 
other out." It is' just this argument that Glass (l 9 77)usedin 
recommending meta-analysis of -weak-studies. While Glass was initially 
confident "that his method' couTd be used with quasi -experiments, his 
vUws have gradually chan S ed (cf. Class S Smith. 1979). The examination 
of the desegregation quasi-experimental studies presented in the 
following sections indicates that selection is a persistent "plausible 
rival hypothesis." That is. it is not cancelled out. Therefore, a 
number of steps have been taken to deal with this. First, an adjustment 
was developed for reducing the bias due to selection. Second, studies 
that were judged a priori not to have selection problems were compared 
with those requiring adjustment. 

The focus of this paper is on the effect of school desegregation on 
black achievement. While interest In these dats is primarily 
methodological am* .terns from earlier work by,,, th. ^author ^ on the 
secondary analysis of the Riverside School Study (RSS) of desegregation 
(Linsenmeier 4 Wortman. 1978; Aoskowit* * Wortman. l 9 8l) . a number of 
substantive issues are addressed. In addition to estimating the 
overall effectiveness of desegregation, such issues as the impact of 
type of achievement (math or verbal) and time of desegregation (early or 
later grades) are also discussed. This latter, substantive focus 
qualifies this study as an "integrative review" (Jackson. 1 9 80) . »n the 
next section, the meta-analytic method used in this study is described. 
As the "disadvantages" column in Table 1 indicates not .11 studies are 
suitable for meta-ana iys i s. Those with numerous' or severe 
methodological M»*. ! adequate reporting of statistical information, 
or insufficient control oeta were not included. In the third section. 



the procedure for including studies in the analysis is described. The 
results and conclusions are presented in the last two sections. 
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2.0 METHODOLOGY 

To apply meta-analysis to quasi-experimental data one needs to 
obtain a measure of "effect size- US). The basic equation adopted from 
Cohen (1969) is: (* £ . * c ) 



ES - — 



(1) 



where, 



% l Q « the means for the treatment 

(i!e.. desegregation) or experimental (E) 

and the control .(0 or untreated (I.e., segregated groups ! 

S « the standard deviation of the control group 

in th, <,u«i-,*P«ri,i*nt»l c,« « the '0<'»- !n S : 

h 2 s ci 



where. 



1,2 indicate 



time 1 (pretest) and time 2 (posttest) 



. 5 X r vieldino Equation 1. However, 
In a randomized experiment. a^, c i » y ,cia,nB 

this .assumption is not guaranteed in a. quasi-exper iment . In this 
.(tuition it is likely that the groups will differ initially. That is. 
section is a major threat to val idity that is represented in this 

model . 

neta-analysis involves summing of the effect size estimates from 
all studies. We define it as: 
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B 



EES « I 1 



s 2i hi 



where, 



X is the sample mean of the experimental or control group at 
■ time 1 and 2 for the i th study and s is the control group 
standard deviation. » 

a is usually presented. This average can be 

The averatje effect s.M. A . .* usuai iy ¥ 

«^ r. r « W u. .n ES . «o - - 

». - — *~ ' l - "J ' 

- .-d n..ee (1982) reanalyzed. Smith ind GUI*' 0977) 
reason th»t Landman and Dane, vsoti i 

mata-analyal. of th. effectivene,. of psychotherapy. 

Th e ,...,r.,.ti.« -itaratur. U Ur.a.y composed of ,u.a- 
e.periment, ~ -n mora poor,, d., S ned .-.«• U H 

. tt .c.pt,P,. t Uty of throat, to <nt.ro., v.Udity «■,... . • 

. 6! „ty to ,nf.r c.u..„ty,. .t r.*» to ...urn. that th... potent, , 

•' .our... of V... «n » tr.at — arror. that are aeW- 

c .„c.,„n, Two threat, in partic •« - - — ' 

reviM , of th,. m-ratura. Th.y ar. "e.Ucti - 

- 9rwt ,.-.r "m.tur.t,on». Th... ire oon.id.ro. ,n tn. neat paraph., 

»r» discussed in the next section-, 
other threats to valid. ty are a.scussco 
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Selection 

Campbell and hi i associ ates (Campbell 6 Eriebacher, 19&7; Campbell 
t Boruch. 1975: Campbell 6 Stanley. 19*6i Cook fc Campbell. 1979) have 
been concerned with the recurrent problem in estimating program effect* 
when various selection procedures are used. In particular, they have 
discussed selection of those students with extreme (pretest) scores and/ 
or matching experimental and control subjects by (pretest) score. Both, 
of these selection procedures are subject to substantial "regression 
artifacts" resulting from the unreliability of the measures used. While 
there is no agreed-upon procedure for adjusting for these selection 
effects, a number of methods have been developed (cf. Wortman. 
Reichardt. & St. Pierre. 1978). These methods require both student- 
level data and test reliabilities in order to be applied. That 
information is generally not reported in the studies of desegregation 
and would require raanaly.L indlyiduaUstudiet if •veiUble. 
instead/the pretest adjustment procedure described in Equations 2 and 3 
will be employed. Since matching was rarely used, this method should 
- adjust-for tnr.c1e.ctJM *f "subject equivalence" problem that Bradley 
and Bradley (1977) and St. John (1975) found to be the major 
methodological weakness in the better or "well designed" studies. 
Neither Crain and Kahard 0982)' nor Krol (1978) attempted to corrector 
adjust for bias introduced by initial subject nonequivalence. 

Differential Srowttv 

It is well-known that blacks and whites show different rates of 
intellectual growth. Thus differential growth or"maturation" may be 
considered an' important". our ce of bias 'in synthesizing the data from the 
desegregation literature. This problem is dealt with -in three ways: 



c.n«ptu,..,. »piri«..r .«* ono.ytie.iiy. " udi " usins 

block control, -or." oxot.ir.od. This I. th. eomp.ri.on recommended by 
St. John (1975) ond .hould roduc. or oli»tn.t. th. problom. Such 
control, ovoid problems (or confound.) c.u.od by r.ce ond .ocioeconomie 
.totu.. They .1.0 .Ho- ..-.motion of th. major policy suction beinp 
.ddf....d. th. off.et of continued reel. 1 i.ol.tion or ...r.o.tioh. 
Fortun.t.ly. moot .tudl.. u.od such . control proup (i.e. ..Seated 
b..ek.). A. noted .bove. both Cr.in .nd neh.rd (1982) .nd Krol (1978) 
included studies th.t used -hite control.. 

Second, th. r..u1t. of th. protest .djustm.nt or. colored to 
tRoS. .tudios not r.ouirin, .uch correction, (i.e.. no protest 
difference.) to determine if other differences or .ources of bios 
rom.in. A. -m b. noted, "differenti.1 repression to th. »..n" (took £ 
Cmpo.ll. 1979) ».y «»unt for th. re.idu.l difference. And third, the 
.n.mic method 1. ...mined to determine it. rcbu.tne,. to thi. source 
of bi... It m.y be reco,ni»d th.t Equotion 2 1. identic.l to th. model 
for diffcrenti.l pro-th r.to. l.belled by Cmpbell the "f.n .pre.d 
hypoth.si." (Cmpbell * Erlebeeher. 1970; Cook * C.mpb.1 1 . 1979) . 1" 
»,et. if di»f.r.«ti.1 pro-th 1. the only c.use of ch.np. from time 1 to 
time 2. then occordinp to the f.n «pr..d model! 

X c - l r • i! c - *r - 



s, s 2 



Thi. hypothesis implies th.t .n lnero.se in the me.n I. occomp.ni.d by . 
proportion.! incrcs. in the -ithin-proup v.ri.nc. Thus. ES • 0 -h.n 
thi. »thr.« to velicity" (1.... diff.r.nti.l 9 ro-th) is pr.sent. This 
me.n. thot ..loction-m.tur.tton Int.r.ction -11 1 not bi.. the ..timet. 



of effect size for quasi -experiments of this type (i.e.. the 
nonequivalent control "group design or NECGD) that are pretest-adjusted. 
This is exactly the model proposed by Campbell 097D and described by 
Kenny 0975). As Campbell and Boruch" (1975) note, standard? ting scores 
will eliminate this problem. The effect size measure as defined above 
in Equation 1 is a standardized score. 

Practical Limi tations 

There arc a number of problems in translating this small analytic 
model into an actual meta-analysis, first, the NECGO requires the means 
and standard deviations for the experimental and control groups on both 
the pretest and posttest. Often these essential data are not furnished 
especially in those cases where statistically non-significant results 
were obtained. The reliability of the tests used is even less likely to 
be reported. In order to deal with this situation, a variety of 
indirect approaches have been proposed (cf. Class. 1977). 

Using Si a nificance Results . Reports often provide only 
^formation on sample size, significance level, and the value of the 
test statistic. In these cases the effect si te can be obtained using 
indirect methods. In the case of the t-test. it is: 

from t * \ - *c 
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where ^ - n 2 and thus about half, of the degrees of freedom (df). then 
according to Rosenthal (1978): 



AM 



This indirect estimate will be conservative when the exact 
significance level is not reported, end the t value is not B iven. 
Typically, the .05 or .01 significance levels are used in social science 
research. If the results are not significant, little if any information 
is usually provided. In this case, a .50 significance level will be 
used as Cooper (1979) has suggested. This Is the expected mean value of 
the distribution of non-significant studies. Simi Lr_Jndirect 
computations can be derived from other test statistics such as I (see 

Appendix 7 in Smith. CUss. 6 w511cr « 1980) * 

Cain Scores . . Another common form of reporting results is the gain 
•core. This is the change in each group from pretest to posttest. In 
Figure 1 this would be: 

gain*£ 2 ~*l * nd C 2* £ T ----- ~. 

for experimental' and control groups, respectively. A simple algebraic 

manipulation reveals that the difference in the two gain scores is 
"equivalent to the numerator in the basic equation to estimate the effect 

size for quasi -experiments (Eq. 2). Thus if t,-t 2 . gain scores can be 

used to derive d for the NECCO quasi -experiment. 
' other OM^ ; . g , aC rimental ••^iiflns. Other quasi-exoer imental 

designs are often encountered and it is important to consider them as 

well. The most frequently reported is the case study or in Campbell and 
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Hypothetical Results From a Study . 
Using a Nonequi val ent Control Croup Design (NECGD) 




E«Experi mental Croup 
C-Control Croup 



St.nUy'. termino.o.y. the One-Group Pretest-Posttest (OGPP) Desi.n. 
This 1. the NECGD without the control .roup. Krol (1979) «u. 9 e S t, th.t 
.„ effect ,Ue e.tim.te c.n be obt.ined by u.in, the protect men end 
st.nd.rd devi.tion ec the control ,roup. Thi. 1. • risky ..sumption in 
our opinion, ond one th.t i, likely to lo.d to en ov.re.tim.te of ES. 

c.n bo re.dily .een in Fidure 1, the use of the st.nd.rdi.ed ,.ln 
•core (Ej-E,) cont.in. . p.eudo-ef feet e,u.l to C } -C,. Moreover, if 
strict .election criteri. .re used .s they often ore in compens.tory 
educfion or competency testin, re.edi.tion pro,r.ms. then regression 
effect, will .1,0 be incorrectly included. Thu, w. fee, ,uch c.se study 
d.t. .hould only b. useiwhen the proper .djustment. c.n be m.de. In 
order to ex.mine design effect, in „.t.-.n..y.L. . «"«• 
studies "ere included in some of the .n.lyses. 

Control group d.t. .r, frequently difficult to oot.ir. for 
politicl .nd pr.ctic. re.sons. rr.gr.rn. m, be designed to serve ... 
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Figure 1 



Hypothetical Results From a Study . 
Using a Nonequi val ent Control Croup Design (NECGD) 



Test 

Score 

Mean 



Time 



'Experimental Group 
•Control Group 



B „,.y. terminology, the One-Group Pretest-Posttest (OGPP) Design, 
is „ the NECGD without the control group. Krol (1979) suggests that 
effect size estimate can be obtained by using the pretest mean and 
■ndard deviation as the control group. This is a r is*y assumption in 
,r opinion, and one that is likely to lead to an overestimate of ES. 
» can be readily seen in Figure 1, the use of the standardized gain 
:ore (Ij-E,) contains a pseudo-effect equal to yC,. Moreover, if 
trlct selection criteria are used as they often are in compensatory 
ducation or competency testing remediation programs, then regression 
ffccts will also be incorrectly included. Thus we feel such case study 
,ta should only be used when the proper adjustments can be made. In 
, rdcr to examine design effects in meta-analysis. * number of these case 
itudies were included in some of the analyses. 

Control grouo data are frequently difficult to obtain for 
political and practical reasons. Programs may be designed to serve all 
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True Experiments . Although our focus has been on quasi- 
experitnents. "true" or randomized studies would be useful . Just as we 
were concerned about the biased estimates produced by pre-exper imental 
design (i.e.. case) studies when compared tc the NECGD quasi- 
experiments. it is important to determine the bias resulting from the 
Utter designs. This information can be obtained if effect size 
estimates are available from randomized studies. Not all data sets have^ 
this mixture of designs, especially in education where there has beSn a 
strong tendency for applied, field problems to be approached quasi - 
experimentally- while laboratory. 'theoretical issues hove been... 
investigated using randomized studies. There have been a few randomired 
studies or true experiments in the school desegregation area. Those 
that have been conducted such as Project Concern (Iwanicki 6 Gable. 
1978) often report their results in such o way as to make it impossible 
to derive effect size estimates. 

Crain (1,983) identified five randomized studies among his top 20. 
three of which were based on data from Project Concern. Three of these 
studies (Rock et .1.. 1*8; Samuels. Iffl. Zdep. 1971 " Appendix A) 
*ere included among the 31 found acceptable in the present analysis. A 
mo re recent report from Project Concern (Iwanicki S Gable. 1978) was 
included in ploce of the two earlier reports used by Crain. ■ 

Pes ton Qua! i ty 

Although the focus is on the NECGD. the quality of the studies 
using this 'design varies. Moreover, as noted above, there are often 
other designs employed. A number of approaches to assessing quality 
ha ve *een developed. The most well-known is the vol id ity approach 
developed by Campbell end Stanley (ISfiWjind recently further refined by 



Cook »d Ce-pb... (.979). E «enti.nr. the thr..u to v.Hdity indict. 
,u,mr. Other. (Borueh £ tatt. .977= Sechre.t * Ye.ton. 1981) h.« 
,t,..,.d the "i.p.-.ent.t.on- or "integrity" .f the treatment. TM. is 
.„ ..portent concept ,.thou g h one th.t i. dlfflc-.t to ...eure. The 
ti ,...*ent of re.e.rch q -.Uty .. . new ere. end one that .. crlt.ee. in 
th. .ynthe... of .c.ent.fic .tudl... There he, teen much di.cu„.on of 
thi. i..u. (nmfl.M '". Bu..e. .977; Er«nc k . .976: G.e.e, .977. .978) 
„„d th. deP.te.ti.. continue. (C. Worsen. lH» • »• the fo..ow.n 9 
section indioet... d...»n due.Uy i. viewed ...n.f.e.nt In .e.eet.n,. 
coding, end en.ly.ing the dot. in . re.e.rch .ynthe.U. 
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3-0 PROCEDURE 

The »«,-.n.1,.l, .PPro.ch first r.,uir.s tM r.tri.v.l of r.lev.nt 
.ci.ntificinfom.tion. TM l.port.«c. of . thoroughly documented 
procedure ,t tM. point h.s b«n .tr.ss.c * both Ccop.r (1982) end 
J.ck.on (1980). To th.t end, «. obtained the cor.per.tion of the euthors 
of the two m.Jor studies .,.t«..t...ll, synthesis, the liter.ture on 
tn. effect, of school desegrcg.tion on bl.ck- .enlevement (Cr.in t 
Mk.nl. 197B; Krol. 197B) . Both Robert tr.in end Ron.ld Kro, generously 
provided copie'. of the .rticl.s .«d the coding schemes u.ed in their 
,n.ly.es. W. then extended .nd updated thl. dot. b... through 
, nocture ...rches including ERIC dl...rt.tlon .b.tr.ct.. references 
|„ the .rtiel.s end book, (e.pecl.lly, St. John, 1975). .nd do«ns of 
,.tt.r. to .uthor. .nd .chool dl.trlct office. We developed . coding 
.ehemc .nd li.t of .tudle. to be included in our .n.ly.es. Thee, ere 
scribed below. As we progressed with our initial coding effort. -. 

,.„,»« thr. * re m.ny studies th.t would heve to be rejected. 

We felt it -.,, -iv. to describe these studies end our re.sons for 
Ejecting th« from the .n.ly,is. W. did this for two re.sons: (.) 
thi . is p.rh.p. the most import.nt. but Judgment.!. ,t.p in d.t. 
synthesis, .nd (b) it is .mport.nt « determine whether there .re uni<,u, 
ch.r.cterieties of excluded .tudies. All studie* r«d .nd coded by 

. t „o independent reviewers. All di.cr.p.ncies were resolved so th.t 
perfect .grecment w.s re.ched. A more defiled description of thi. 
procedure .nd the .tudies exciuded din be found in .n e.rlier technic.! 
report (Wortm.n. Kin, . Bry.nt. 198M.' In the n.xt three -sections w. 
discuss both of these concerns. 
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Exclusion Criteria . The decision to exclude a particular study 
from the analyses was based on assessments of the various threats to the 
study's validity. The number and magnitude of the flaws in the study 
were the dec id in B factor for inclusion or exclusion. The observed 
threats to validity fall into one or more of four basic classifications 
that have been developed by Campbell and his associates (Campbell 6 
Stanley. 1*3; Cook I Campbell. 1979). Thus, the criteria used to 
reject studies (see Table 2) represent specific instances or threats to 
internal, external, construct, or statistical conclusion validity. 

eternal validity is broadly concerned with whether the treatment 
(i.e.. school desegregation) in fact affected the outcome (i.e.. 
academic achievement of black students). Threats to internal validity 
m ay be posed by uncontrolled variables representing effects of history, 
juration, and the like as originally described by Campbell and Stanley 
0 9 63). nost of the factors listed in th* table as threats to validity 
do not require further explication. However, the rationale behind a few 
may not be so apparent. For instance, studies utilising cross-sectional 
survey-designs (criterion *.) were rejected from , the analyses because 
they typically do not control for extraneous variables in local school 
settings that may affect achievement above and beyond the effects of 
desegregation. That is. they »re usually observations. at one point in 
time lacking both pretests and adequate controls. 

Studies were also rejected that failed to describe their sampling 
procedures ' (cr i terion ^ ' and thus make it impossible to rule out 
potentially confounding biases in the selection of comparison groups. 
F5nally . the use of different tests for segregated and desegregated 
students.! either pretest or posttest may pose "instrumentation" 



■ table 2 

Criteria for Selecting Studies for Neta-anUysis 




Criteria for 1 Reliction 



1) lYES 9l IMV 
♦«) NOM^tuplrlcfll 

♦b) Supary report 1 : 



Insufficient detail for coding 



2) location: 
♦n) Ouhlde U.S.A. 
♦b) Olograph t rally non-specific 

V) NoTnMy of Achievement of desegregated Blacks 
•b) »il H -ethnic data combined 

♦c) Comparisons across ethnicities only rnnrt mon 
»d) Heiopww ^portion minority In desegregated condition 
♦e) No control or pre-d»segregation data 
♦f) Control mrasurps not contemporaneous 
p) Multiple treatment interference 
h) fotwilva attrition 

♦l) NalwHy black In desegregated condition' 
♦j) varied exposure to desegreg.it Ion' 
k) Croups initially non-comparable 

4) Study. Doslfi:' 

»a) Crnss •sectional survey 
•b) Sampling procedure unknown 
"«e) Sep* at* iion-crmpatflble samples at each observation 

d) Grad* lav«lo grossly combined 

e) tnadniunte sample sue 

91 llnSfSi Imble and/or unstandardUed Instruments 
•bj lest content unknown 
«»:) Oale-i of administration unknown 
♦d) Different teals used at pretest and posttest 
test of 10 or verbal ability 

6) Data Anoiysta: 

; a) No pretest means 

♦b) No post lest means 4 

»c) No nietest standard deviations' . 

♦d) No rustiest standard deviations' 

•a) No niqnlf icance tftsta 

♦fj No rti»a reported 

♦g) m * HiscifHAbtr' 

h) liwiH«nnr»ate statistics 



Internal 



External 



Construct 



Statistical 



♦Criteria us».t in wimel NtF Core Studies d t m . 8peclflc justification' for this. 

4p s s s: sirs s: satST^ry-* — » *»»■ «-* siw - 
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problems stemming from differential test reliability and low inter-test 
reliability. These .problems may either produce spur lous treatment 
effects or mask real effects. Each of these specific threats may 
confound the observed association between desegregation and achievement. 

External validity refers to limitations in the general irabi 1 i ty of 
the study with regard to populations, settings, as well as treatment and 
measurement variables. One obvious reason for exclusion was studies 
conducted outside of the United States. Another common threat to 
external validity involved the confounding effect of compensatory 
equalization of treatment U.g:. extra teachers for segregated. controls). . 
or other kinds of multiple treatment interference (criterion 39> . These 
may disguise or distort findings indicating how desegregation affects 
achievement. Moreover, when the dates of test administration are not 
described (criterion 50. problems arise i« adjusting the effect-size 
estimates to a proper time interval as well as determining whether the 
pretest actually occurred prior to desegregation. 

Construct validity, refers to the appropriateness of the 
theoretical constructs, variables, and measures used. If the study did 
not really deal with desegregation and/or achievement, it was not 
included. Other studies were rejected on these grounds, but for less 
- obvious reasons. These include those that at first appear to measure 
academic achievement of desegregated blacks, but which, in fact, measure 
. different construct such as I.Q. (an abi 1 ity measure) : those that 
"measure a ' dtf f erefif 'V»t«Ht. ««eh as bus transportation; or a 
different population such as whites or Chicanos (see criterion 3») . 

Statistical inclusion validity is concerned with the 
appropriateness of ' the .\atisti c.l analyses. This includes not only the 
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analyses employed but also the sufficiency of the data reported for 
calculating effect sires. For example, a study may improperly use ANOVA 
in the analysis of a non-equivalent control group design (i.e., 
criterion 6h> that violates assumptions of homogeneity of variance and 
of heteroscedastici ty. Other studies may correctly employ statistical 
procedures where there is inadequate statistical power from sample sires 
"too" small to reject the null hypothesis. Finally, studies which grossly 
combine achievement results of different grade levels must be rejected 
because the rate of achievement gain tends to increase more slowly with 

t 

advancing grade level and thus grade-equivalent scores are really not 
comparable (as they are normed within each grade separately). Combining 
scores from various tests across grade levels further threatens internal 
validity insofar as instrumentation effects ari se from variations in 
test reliability and other test characteristics (e.g.. item difficulty 
and content) . 

Applying the criteria listed in Table 2 resulted in the exclusion 
of 7i» studies. Most suffered from more than one problem. A number of 
tfteac critaria arc sufficient in themselves (i.e., "fatal flaws") to 
eliminate a study. All but three studies had such flaws. Overall, we 
have had to exclude the majority of studies examined including a number 
•^usedln'the previous meta-analyses -performed (Drain 6 nahard. 1978; 
Krol, 1978). A comparison of studies included and excluded is provided 
in Table 3. With the exception of Crain and nahard (1978), we included 
only about' half of the studies 'used in other major reviews. The 3^ 
studies included in our analyses are listed in Appendix A. The studies 
were decomposed into effect size data for each graae and for reading ano 
mathematics achievement, and thus yielded 106 separate "cases". The 
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overall analyses, however, used the study as the unit of analysis by 
averaging the results within each study and combining these average 



effect sizes. 



Table 3 

Comparison With Previous Research Syntheses 

% of PRESENT CASES USED BY PAST I NVESTf GATORS 



....... PRESENT 

CASES 


KROL 


CRAt-N -fr- - 
KAHARD 


1 

WEINBERG 


ST. JOHN 


(REJECTED (n-229) 






25* 


2a 


ACCEPTED (n»106) 




m 


51* 


57% 



A considerable amount of 'effort was spent in documenting this 
aspect of the research synthesis. It represents an important, but often 
overlooked, part of formal data synthesis procedures, and one that can § 
produce differing results. Whi 1c meta-analys-is. itself, is a formal, 
quantitative method, the selection of the .ample to include in the 
analysis is not. ^ without appropriate, documented selection criteria, 
the .results, -can. M,a$ tubjectiye. and. biased _os the 1 1ter ature reviews 
they seek to replace, (cf. Jackson, 1980) 

One "disadvantage" of meta-analysis (see Table 1) is its 
susceptibility to publ Heat ion bias. It Is assumed .that the research 
literature contains only studies showing positive. statistically 
significant results (i.e.. publishable studies) . The 1*1 studies found 
"acceptable'' contained on Vy^; W >ub 1 i shed articles. ^Desegregation 
research is largely (and perhaps appropriately) . a fugitive literature. 
We feel that the retrieval strategy described obo r ? v eapturec tne 
••target population" of studies (Cooper, 1982) . 



The Nl£ Core Studies 

After thi. .creening proof hod been performed ond the 31 
re.u.tin, .tudie. .nal,»d. the NIC D.f egrcg.tl.n Studio. Too. convened 
,n expert panel to .elect the be.t. . ftudi.f in thi. .re.. The p.nei of 
,ix .ehol.r, including thi. .uthor -af .uppofedl, b.l.nc.d in their 
.ttitude. .nd publi.hed work on desegregation -- t-o pro. two con. .nd 
"two' n.utr»..«: Th. p.nei met in Jul,. 1 9 82 .nd initi.ted discuffion of 
the mo.t .ppropri.t. .tudie. to be included in revlc-in, the literature. 
The criteri. lifted in T.bl. i were examined b, the p.nel .nd .ft.r some 
dUcufion . .ub,.t of ih»«a. -mU-U f.1«t th. highe.t «u.litr 
.tudi.. available m 9 .n.r.l the,. w.r. NEKD ftudi.. comparing verb.l 
„d/or ..th .chi.vw.nt of de.egr.g.t.d .nd f.,r. S .t.d bl.ckf. The 
criteria .ctutlly ufed are .tarred In the t»bl«. 

Th... criteri. w.r. entered into the computer i«.d data b.fe .nd IB 
.tudie. -.re found th.t ..ti.f i.d the.. r.,uirem.nt,. Th... .tudie. .re 
.tarred in Appendix A. One new .tudy b, W.lber, (1S7» «« » *• 

r .,u..t of ..« of th. p.n.l member.. Thi. .tudy h.d b.en "r.j.cted" in 
the *rl,l«1 analr." .i«« it .uf fered f ro» ." •»«*»>» high rate of 
attrition (criterion Jn) th.t differed for .egrcg.t.d .nd delegated 
.tudent. (i.e.. » - "* r.^ctWel,). Th. number of 

•itudihtrih-th. dcegregatef control .roup ... quite .mail, ranging from 
Uto53. nor.ov.r. grade level, -ere combined (criterion M> . The 

w.iber, ftud, added eight "ca.e." to the d.t. b.fe. «oreov e of 

t ne'p.nell.t.-r.t. too»ne 6f tfi.-.at«or rf another atud, (Sh.chan. 
,579, to obt.in .i..i«g ".nd.rd deviation,. Thi. .11o-.d the 

inclusion of t-o »dditlon.l c.tet. 



Th... .tudie. differ .ub.t.nti.lly fro* tho.e used in most previous 
review.. With the exception of Cr.ln end M.h.rd (1978) -here .11. but 
one. study -.. includ.d. fewer th.n h.lf were included in prior review,, 
for ex. m ple. .Br.dl.y end Br.dley (1977) ineiuded only five of these 
studies while St. John (1975) reviewed only nine of them. 
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k.O RESULTS 

Tnc 6U „ effect si«s USs) for the 31 studies considered 

Mthrt.1i1e.11y acceptable for performing . meU-.nalysU-.re presented 

In Table k. The' fourth row "labelled "Grand" present, the-overall 

effects averaged by study (i.e.. the average of the average effect sires 

for each study) and the ESs by three major research designs. In 

addition, these four categories are broken down by grade in the bottom. 

twelve rows. The ESs for reading and mathematics are combined in this 

initial analysis to provide a single measure of overall effectiveness. 

Since some reviewers have noted greater gains for mathematics than 

re 1075- Krol. 1 Q 78) . ESs for these two areas 

verbal achievement (St. John, 1975. *roi. 

of achievement were also examined and are reported below. 
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GRADE 
LEVEL 
AT 

POSTTtST 




PnOLEO 
TOTAL 
OF 

■ACCEPTED* 
SAMPLE 



1 


2 


2 


10 


3 


ft 


4 


IT 


5 


22 


6 


15 


7 


4 


0 * 


2 


9 


5 


10 


4 


II 


4 


12 


3 



, 4 

r ♦ 



0.19 (0.01) 
0.17 (0.11) 
0.39 (0.71) 
0.44 (0.54)' 
0,51 (0.89) 
0.5fi (0.86) 
.98(0.19) 
.80(0.34) 
0.02 (0.07) 
13 (0.03) 
12 (0.09) 
■6.15 (0.01) 



0. 
0. 



M11,99W,91, 
g < .005 



Ona Group preteat-PoaWeat: 




0 
1 
1 
2 
3 
I 
2 
2 
0 
2 
2 
2 



0.01 . 
2.15 - 
2.03 1.20) 
1.54 (6.22) 
3 19* 

a" is jo.ii) 



1.80 (0.34) 

0.00 (0.29) 
0.15 (0.13) 
•0.13 (0.00) 



F(9,17)«1.19, 
n.a. 



Nonequlvalant Control Groups 
0 X 0 



1 


-0.24 


5 


0.09 (0.07) 


5 


0.28 (0.25) 


9 


'0.39 (0.10) 


16 


0.38 (0.17) 


10 


0.18 (0.33) 


2 


4 1.79(0. 30) 


0 


• * 


2 


0.10 (0.20) 


2 


0.25 (0.01) 


2 


0.09 (0.01) 


0 





FI9,93)»3.24, 
" E < .01 



Static Group Comparlaon: 
X 0 



1 

2 

0 

6 
t 

w 

4 
0 
0 
3 
0 
0 



•0.14 
0.08 (0.29) 

•0.03 (o.o?r 

0.17 10.00) 
•0,67(0.16) 



•0.03 (0.02) 
•0.18 



M6.1SH.B2, 
B<.01 



observation* per study). . . 

V.,, ^ «. «*•.». «*« •»«'••«• 6y " '"* le " IM,in Meh ,,u * Y "" ! 1 " 



0.42. 

e 



fc. .„«,,.» NT on. NMW ** "f.« W * ' W * " "T* * • 000, 

. . .... i «n r • H.47. df»2.9l. D 9< .0001), 



The overall ES for the 31 studies is standard deviations. The 
ES is relatively unaffected by various weighting schemes. This figure 
is considerably larger than those reported by Grain and Kahard (1982) 
and Krol 0978)'. However, the ESs for the more well-designed quasi- 
experiments are considerably smaller (i.e.. .32 and .18). St is clear 
that the studies using the weaker OGPP design are inflating the estimate 
of the ES (i.e.. 1.22). As was" noted earlier, this latter design 
confounds maturation and initial differences in student selection with 
the! effect of desegregation. Such design effects resulting from 
differences in study quality are commonly reported (cf. wortman. 1983). 
in practically all such cases the weaker designs produce larger 
estimates of effects. Thus design quality must be considered in 
conducting an integrative review. As Jackson 0980) notes. -The results 
of the analysis may be misleading if there is not at least a modest 

number of studies with good ovcraU,. design." , .. .. 

The bottom twelve rows of the tabic present the results by grade. 
The general pattern is for an increase in ES for grades 1-8 followed by 
a'declirie-forthe later-grades. This findiag contradicts, those reputed. 
by Grain and Mahard (1978) and St. John (1975). The Class ES for grades 
K-6 was slightly, but not statistically, lower than the ES for grades 
" 7-12 '(.i.3 »nd .55. respective^)/ "Given the varying duration of these 
studies. Stephen (1982) calculated the ES per month" for the N1E Core 
Studies. He found a pattern consistent with Crain and ftahard (1982) and 
St. John (1975). 

All of these estimates of ES are susceptible to bias due to 
selection or absence of initial subject equivalence. The result for 
those studies where it was possible to employ the pretest adjustment to 



remove initial differences between segregated end desegregated groups 
are presented in Table 5. These studies used the non-equivalent control 
group design and reported sufficient pretest information to calculate 
ESs. 

Table 5 

Adjusted and unadjusted methods for the 
meta-analysis cf quasi -experiments 



Computation 
Method 


Overal 1 
Heart ES 


Selection 
Problems 


No Selection 
Problems 


Unadjusted 

Pretest 
Adjusted 


0.U2 (n«32) 
0.16 (n«32) 


0.57 (n-20) 
0.16 (n«20) 


0.20 (n«10) 
0.20 (n«10) 


Pai rwi se 
t-value 


t &2 «2.73. £ < -02 


l 3 9*2.9Ji. £< -01 


lis* 0, n *** 



a ln two cases it was not possible to determine whether or not there 

•i 

were selection problems. 



The first column of the table indicates a sizeable and 
statistically significant difference between the "overall" unadjusted, 
Class effect-size estimate and the pretest adjusted estimate (.U and 
.16. respectively). The Class estimate is similar to that reported 
above in Table k. All studies were initially coded along a number of 
dimensions including most of Cook and Campbell's threats to validity 
before any effect sizes were actually calculated. The second and third 
columns compare studies with and without selection problems. The Class 
£S estimate is higher for those studies with "selection problems" than 
the overall ES while the pretest-adjusted estimate remains the same as 



before (.57 and .16. respectively). Again, the two estimates are 
significantly different by statistical criteria. On the other hand, 
where selection was not considered a problem, the two estimates of ES 
are exactly the same (.20). This number is si ightly higher for the 
pretest-adjusted estimates since two cases were omitted where it was not 
possible to determine a priori whether selection was a problem. 

The difference between the pretest-adjusted ES and the ES for 
studies without selection problems may result from differential 
regression. Since the students involved in these studies generally 
.core below the mean for their grade, their scores will regress to the 
higher mean at post-test solely due to the measurement error in the 
tests. Moreover, with an initial difference of .26 standard deviations, 
the control, segregated students will regress more. This implies that 
the pretest correction overadjusts slightly. Assuming a reliable test 
reliability of 0.8 to 0.9 for these students wi 1 1 account for the .014 

difference. 

The pretest-adjustment method thus appears te remove the initial 
differences due to subject none^ui valence, it is the author's opinion 
that this provides a fairly accurate estimate of the overall actual 
benefit of desegregation or. minority, black achievement. According to 
Class et al. (1981. p. 103). each .1 ES is equal to .1 grade equivalents 
or one month of educational gain. Thus desegregated students may be 
gaining about two months due to attending an integrated environment. 
The analysis indicates only a slight, but statistically non-significant, 
gain for the few cases where results greater than one school year were 
reported. Similarly, there were only a very few cases where the 
percentage black was reported. When the difference between percentage 



buck in the control (i.e.. segregated) and treatment (i.e.. 
desegregated) groups' was calculated, it revealed that most of the 
effects were obtained in those studies where the difference ranged from 
7b to 85 percent. That is. students moving from almost completely 
segregated environments to predominantly white schools showed a sizeable 
(1.0b ES using the Glass method) effect. This finding is consistent 

with the Coleman Report. 

Finally. the Glass effect size estimates for reading and 
mathematics were examined separately. These results are presented in 
Table 6. As with the overall ES. both effects are positive indicating a 
benefit for desegregated students. Contrary to previous research (Krol. 
1978 ; St. John. 1975) the ES for reading achievement was considerably 
larB er than that for math (.57 and .33. respectively). This difference 
was not statistically significant, however. Thus a si ngle overal 1 
estimate of achievement effects appears to be an appropriate measure of 
the impact of desegregation. 

Table 6 

gean EHe^Size For flath Vs^ EiMiUfl kzhl£^S^ 



Achievement 
Measure 



rtath (n«37) 
Reading (n«5D 



Mean Glass 
ES 6 (o*) 



0.33 (0-3B) 
0.57 (0-9 1 *) 



1.86. dfi.87. £ < .18 



Note-Krol found a tendency for math achievement to show a greater 
effect-size than reading achievement (t^-1-90. p-.OB) . 
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The NIE Core Studies 

A similar analysis was performed on the 19 studies selected by the 
NIE panel of experts. The results are presented in Table 7- The 
information is presented by study with overall effects presented at the 
end. The pattern of results is quite similar to those presented above. 
All ESs are again positive indicating a beneficial impact of 
desegregation on achievement. The ESs are slightly lower partly due u> 
the inclusion of the negative ESs for the Sheehan (1979) and Walberg 
097D studies. » 

The overall mean unadjusted Glass ES is .25. The unadjusted ES 
estimate is comparable to the .23 reported by Crain and ftahard (1982) 
and. more recently, the .2* by Crain (1983) for the best designed 
studies. It is only slightly less than the .28 ES that Crain and nahard 
(1982) claim for "the estimated treatment assuming the best possible 
research design." However, all of those estimates ignore the bias 
introduced by the initial nonequi valence of the students. When 
adjusted for pretest differences, the ES is reduced to .U. Compared to 
the original 31 studies, the decrease for the Class ES is .17. but it is 
only .02 for the pretest adjusted ES. The reason for this is that 
negative ESs have been added by the panel to the core studies which 
largely, but not entirely, reflect pre-existing differences among 
segregated and desegregated students. In these cases, however, the 
differences favored the segregated students. In 'act. there is a large 
correlation between pretest and posttest effects sizes (r • .76) 
- indicating that pre-existing differences largely remain at the posttest. 
Thus subject equivalence is a persistent sourc* of oias in these 
studies. It U.for this reason that the pretert adjustment method was 
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employed. This adjusted ES provides a less biased estimate of the 
overall effectiveness of desegregation. The adjustment is equally 
successful for studies with large ESs (greater than 1.0) such as Rentsch 
(1967). 

As with the larger set of 31 studies, the core studies show the 
effects for reading achievement to be modestly larger than those for 
mathematics (.28 and .23. respectively). However, when these figures 
are decomposed by duration or length of desegregation, there is an 
interaction with mathematics showing larger effects for those studies 
longer than one year. While there are relatively few cases available, 
this may explain the difference between the overall results in this 
study and those reported by others. It may be that studies of longer 
duration comprised the majority of those reviewed by Krol (1978) and 

St. John (1975) • 
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TabU-Y. Effect 5I»« for N1E Core Studies 
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5 
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NA 
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4 
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5 
5 
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6 
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NA 


NA 


NA 


NA 


NA 


NA 


NA 


NA 


NA 


NA 


NA 


NA 


NA 


NA 



Achievement E ffect Size 
Be ading Hath 



1.14 
1.27 
2.17 

.01 



-.29 



.42 



.22 



.75 
.33 



.25 



.13 
.11 
.16 
.29 



,34 



Protest-Adjusted 
Effect Size 



.95 
.92 
1.40 

* • 

.17 



-.27 



.49 



.42 



.10 

« • 

.28 
.36 



.29 

.28 
.36 
.06 



.15 



.19 
.06 
.58 
-.17 
,76 
-.22 



.14 

.09 



,16 



-.05 
.10 



.59 
.11 



.93 



.24 
.21 
.01 



.65 
.15 
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Effect Size 




MEAN FOR WWnWffS LASTING 
ONE TEAR OR LE55' 

MEAN FOR TREATMENTS LASTING 
MORE THAN ONI! YEAR' 
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5-0 SUMMARY 

The synthesis of ".dentine research using form,. .tdtl.tiea, 
procedure. »'«»' ».t.-.n.ly.l. present. .peel., P^'»' — » 

.tudie. ar. m.thodologie.Uy flawed. The research llt.retur. en the 
effectiveness of school desegregation on minority bl.ek achievement is 
„»o,t t.t.,1, eempri.ed of que, i -experiments or weaker rc.e.reh 
d.,i,n.. Whi ie CUss he. recommended ine,ud,n, .11 studies in. 
research synthesis, his work ho, largely de.lt with studies that are 
•Veil designed." In the.. Instance, where "poorly designed" studies 
„.v. been included, design effects have been found (Class a Smith. 1979: 
Cilbert.t.l.. 19771 wortman. ,98.) indicating major difference, in 
estimate, of effect, between .tudie. with .trong and weak de.ign.. The 
typical approach to this problem 1. to examine the 
.tudie. t.kin, into account, where possible, the flaw, or threat, to 
validity. Thi, was the approach taken in this study. Specific 

methodological criteria for Including .tudie. in t .arch .ynthe.i. 

were developed and applied to the .chool de.egr.g.ti.n llter.ture. A, 1 

, tuiI . found t ome .erious flaw., but J1 war. considered 

.cc.pt.ble for an.,,.!.. Even within thi. .et. there wa, variation in 
design duality and a con.ider.bU design effect. The Nit panel of 
expert, decided to include on,, the highest duality studies and th.. 

furt „ ced the ... to ,8 .tudl... The .tody by .alder, <>97» ~ 

fit to be of .ufflcien. ,u.„ty to be added to this ..t although It h.d 
er,9ln.11y been "rejected" for . variety of methodological f>.".. 

The N,£ Core Studio, hod an overa,., effect of .25 ."nd.rd 

deviations. This ., .-most identic., to the effect estimate 
reported by Cr.ln .nd „,. ..sedate, for w.„-de„ 9 ned .tudie.. Since 



„.t of these .tudie. .offered from in1t..l subject none,ui valence, .. 

.dju.t.d .ff«t . « «. OeuLted by .ubtr.ct.n. cut the effect .1,. 

.t the pretc.t prior « desegregation. This reeulted in .n effect .i» 

ef Given d.ff.r.nti.1 ,t.ti.tL.l r. 9 r...ion to the ...n. thi. 

pr.b.b„ . underc.tim.tc. Thi, i. .!.!•« to th.t found for the 

Urgcr .et of Jl .tud.e. end .Ho to Kr.,-. (-378) finding, in 
cmining the r..u.t. of the two .naly.es reported ebove. 
ov.r.,1 e.tim.tc of the effect ef school d.se.rcgtion on bi.ek 
.enlevement appear, to be .bout .2 of . .t.nd.rd devi.tion. Thi. 
£ ,tim.t. i. ba.ed on thee «... not h.vino .election problem, .nd 
comparable to the adju.ted e.timatc.. 

Other .ub.idi.r, an.l,.c. eemp.rin, type of achievement, duration 

, *i fi^H difference in percent black for 
of de.cgrcgation. gr.de level, end d.fferencc 

...reg.ted end de.e S r. 9 .ted .tudent. -ere .Uo .«..-. 

found to be .iightly higher then math .enlevement .Ithou.h thi. may vary 

„itn ,en 9 th of de,e,re 9 .tlon. The i.r.er .et of .tudi -ed • 

curviiineer pettern of effect, with an increa.c from gr.de. K-7 .nd . 
oecrca.c from 8-12. Thi. re.u.t doe. not .gree .1* other finding, 
indicting l.rg.r benefit, the e.riier desegregation occur.. No effect 
„. found for .mount of de.egreg.tion (i.e.. "an one ye.r compared 

to more th.n one ye.r,. Some .upport ... found for the findin, of the 
Coleman Report th.t effect, .re .reete.t in the mo.t integrated 
envi ronmcnts^ 

Wh.t do th... findin,. mean? The effect sic. found in both 
analyse, reported here indic.te. .bout . two month gain or benefit for 
segregated .tudent.. The meaning .tt.ehed to thi. findin, represent. 
, . judgment. Thi. I. -her. .oel.l .clenc. end. .nd. .cel.. policy begin.. 



However. ». h.ve «- .ci.ntifie .it.r.tur. on eeron.ry-.rt.ry 

bypass ,r.ft .urgery" for comp.r.tiv. purposes. This is . widely 
accepted -edle.l procedure that I. currently, performed on well over 
,00.000 person. annu.1 ly at a co.t of ne.riy 52 bi 1 1 ion. nuch of this 
expense is reimbursed by third-p.rty pey.r. includin, the feder.l 
government. . r.se.rch .ynthe.1 . of the hi 9 h.r-,u.l i t, studies (i ... 
r.ndemiaed) found . benefit of .8 et.nd.rd devi.tlons rcprc.cntin, only 
„ ... percent incre.se in surviv.l r.tes (WortMn t vc.ton. in press). 
This i. .modest incre.se at . con.ider.bl. .ocl.l co.t -hen comp.r.d to 
.chool segregation. Moreover. program, .imed et th. youn, .uch .. 
.choo, d.s.gr.g.ti.n typic.lly .re more co.t effective then th... for 
the elderly .uch a. bypa.s surgery. 

..though the method, developed above h.ve been u.eful in dealing 
with problem, of .tud.nt equivalence, they c.nnot .dju.t for th. second 
„.jor problem net.d by St. John (1975) of "equivalence *f .chool.." The 
ectue, det.il. of the eduction.! program, involved in th. d... 9 r.g.tion 

, tudl ot reported. Thu, it !. not po..ib,e to determine 

effective from ineffective program.. The real problem a. Ber.rd and 
Mil* (1975) conclude i. "to fo.ter int.gr.tion of the minority 
children W. th. classroom aoci.l .tructure and ...demie program." 
Seeent .tudie. h.ve eddre...d thl. i.au. and developed procedure, for 
improving .duc.tion.l practice in delegated cl...room. (Aron.on I 
B rid 9 .m.n. 1979. Sl.vl B * H.dd.». 1979). » "umber of the paper, .y 
^ber. of the NIC expert p.nel focu.ed on the.e pr.c.dur... Such 
re...rch b.sed on sound soci.l .cl.nce theory I. liKelr to lead to 
increased educetional l benefit, for oesegregeted students. 



The political reality confronting the achievement of school 
desegregation today "l. the need to allow students in highly segregated 
urban inner cities access to schools in the surrounding white collar 
suburbs. Such -metropolitan plans" have been found to achieve 
desegregtion without white flight. They are also quite controversial 
and typically require cross-distr ict busing. The results in St. Louis 
.re encouraging. Here voluntary cross-di strict busing combined with 
inner city magnet schools have produced two-way desegregation with some 
whites returning to the city schools. } It should be noted that the plan 
i. an alternative to court-ordered mandatory metropolitan desegregation, 
moreover, it should be added that such plans resemble the early 
voluntary plans in the Northeast. as a social polity, these plans 
~ capitalising on good suburban schools, a cooperative environment, and 
novated volunteers -- produced the largest effects of the studies 
examined. 
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6.0 FOOTNOTES 

H-.'. «ti».« o* •«•" •«"• *• " id ' nti " , ' T " 

1«- ~- — nt ° nd contro ' sroup ; 

U . H . M fiqB2) maintains that this 
tnc pooled-within standard deviat.on. Hedges (19*2) 

a ~*t~~+ However, this estimator 
produce. . >«.. bios.d ..»<■»« of effect. 

k |_ «u.«i by the effect of the treewent on the 
ignores problems cause »7 

. xp .r,»er.t.. (...-. dc.egre.eted, S roup .tenderd devietion. 

. Mft tQ calculate effect si2es 
i was not possible to cai^u.-* 

* Unf or tuna tel y , 1 1 was noi k 

S „n« problem P»9"' the e.rHer report. .. well. 

.,„ feet, one of the "neutr.." -Per, Md te.tifi.d nu»erou, 
time, «9»>Mt desejrejation in court cdtes. 
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